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ABSTRACT 

1 The high degree of autonomy college faculty members 

exercise in organizing and teaching courses suggests tfiat faculty 
perceptions of educational purposes, subject matter structure and 
importance, and expectations for student learning are major • 
determinants of educational success. Despite diversity and local 
autonomy present in its system, American higher education shows a 
substantial degree of coherence. Students and faculty move with 
reasonable ease from one institution to another. The autonomy of 
individual faculty members is countered by a complex network of 
influences: (1) prior educational and work experience and background 
characteristics; (2) textbooks and other instructional materials; (3) 
direct contact with other faculty members (e.g., curriculum 
committees, departmental discussions, informal contact); (4) 
professional literature; (5) professional associations and learned 
societies;, and (6) the accreditation process. The first section of 
this paper describes the nature of the faculty network and its 
constraining influence on faculty teaching, educational content, and 
standards. The second section discusses how the effects of faculty 
views and expectations on student learning, as constrained by the 
existing system of influences, are or might be assessed. Procedures 
are described that would extend knowledge of the educational 
accomplishments of the higher education system while allowing for its 
diversity. (JD) 
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For 30 years, American higher education has probably been studied 

in greater scope and intensity than has any other educational system. "Set 

information has not been produced that would give defensible answers to such 

fundamental questions as the following: 

o What do the students who go through the American higher education 
system learn? 

o With great diversity in educational curricula and programs, how 
effective is any one curricular or programmatic approach in 
comparison with any other? 

o How good, or how poor, is the system as a whole in relation to 
what it might be? 

An immediate response to such questions might be that they are naive, 
that American higher education is too complex to permit realistic or useful 
answers to such broad questions. This paper will propose that questions as 
broad as those can be answered, although they will not "be answered simply, 
and that faculty members toe^the central actors in determining the questions 
as well as the answers. ^ 

The American College , edited in 1962 by Nevitt Sanford, reported the 
results of an unusually productive decade of research about higher education 
in America. In the 29 chapters by 30 authors, the dominant themes were 
related almost exclusively to student development in the social and cultural 
context of the institution and, to a lesser extent, society at large. 

Little was said about the content or substance of student leaminc. The two 
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or three chapters in which the curriculum was discussed treated it almost 
entirely in terms of its effect on student personality development. 

The slighting of intellectual development in favor of a broader view 
of students' developmental needs was purposeful and legitimate in view of 
the convictions first, that higher education had been too narrowly focused 
on" academic concerns to accomplish them much as well as was desirable, and 

. o 

second, that the personal development of students deserved more atteation 

I*. 

than it had been getting. During the twenty years since publication of The 
American College , the personal development of students has dominated studies 
of higher education. The severe limitations in our understanding o^intel- 
lectual growth have only slowly begun to be widely recognized and have 
barely begun to be sketched. 

Feldman and Newccmb (1967) in an exhaustive review of studies from the 
middle 1920' s to the middle 1960's on the Impact of college on students, 
discussed values, goals, satisfactions, attitudes, interests, and personal- 
ity traits. "Learning* is not in *heir index. References to "academic 
achievement" deal with students 1 attitudes toward achievement and to peer 
and faculty influences on those attitudes. The scope and depth of the 
knowledge and understanding students acquired in college were almost com- 
pletely neglected, without explanation. 

Prom 1967 to 1973, the Carnegie Commission on Higher Education -under 
Clark Kerr produced an extensive, probing series of reports on American 
higher education that received wide attention in the United States and 
abroad. Its impact has been substantial, on federal legislation as well 
as cm institutional practice and public policy. Despite general acclaim t N 



though, it too slighted the substance of higher education. One knowledge- 
able critic objected that the Carnegie Commis-sion "does not inquire into 
what is meant by quality with a rigor equal to that with which it pursues 
... the quantitative" (Niblett, 1973). Another agreed, noting that the 
Commission "spent most of its energy and attention on the arrangements and 
circumstances of higher education rather than on the education itself" 
(McDonald, 1973). Alan Pifer, president of the Carnegie Corporation, 
acknowledged that the Commission did indeed neglect the curricula of higher 
education, but pointed out that the Commission had not been asked to address 
that topic (Carnegie Council, 1980, p. 31). 

The Carnegie Council on Policy Studies in Higher Education, successor 
to the Carnegie Commission, had a similarly productive life from 1974 to 
1979 and gave somewhat more attention to questions of curricula and student 
learning. Appearing repeatedly in the Carnegie Council's reports, though, 
and in statements of others who have systematically observed higher educa- 
tion's functions, is a wistful acknowledgement that little is known of 
students' actual learning as they progress through their undergraduate 
programs. Howard Bowen (1977), after having completed for the Carnegie 
Council probably the most exhaustive search that has yet been made for 
evidence of the broad range of effects of higher education, called the 
evidence about college effects on verbal and quantitative skills "dis- 
tressingly meager" (p. 67). On a„ later date, be commented that, ~the 
residue of a college education— after the initial forgetting of detail— is 
a virtual mystery" (Bowen, 1979). The Carnegie Commission itself had noted 
earlier, "Performance ie very hard to measure in this area [the education 
of individual students] given the millions of students, the thousands of 



^campuses, and, the several and quite imprecise criteria" (Carnegie Commission, 
1973, p. 20). " 
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The following comments, repeating the common theme that we know too 
little about student learning, span the past 15 years. 

[T]he questions of what should be taught, and how and when, are still 
more a matter of mythology than of rational judgment (Boyer, 1967, 
cited in Trent & Cohen, 1973, p. 1033). 

Most studies [in the area of curriculum] • . o . merely note trends and 
describe what is being done . . . but ignore the relationship of the 
trend to the goals of the college .or the effectiveness of the course 
(Trent & Cohen, 1973, p. 1032). x ' 

o Of all the matters treated in this chapter [on undergraduate science 
education] an assessment of the quality of science education and its 
change with time is most, central. Yet our information is almost 
nonexistent: (Doty & Zinberg, 1973, p. ,164). 

By what standards may one judge how effective current teaching and 
' learning actually are and how effective they can reasonably be. expected 
to become? ... Questions of this sort arise because little is known 
about the outcomes of college instruction (Wilson, Gaff, Dienst, Wood & 
Bavry, 1975, p. 87). e 

It would seem an appropriate time to look at what it is English de- 
partments do (Lambdin & Fowler, 1981, p. 335). 

O^e of the problems in identifying with any degree of precision the 
kinds ofXlearning or intellectual growth attributable to college and 
.university programs was described by the Carnegie Commission as the sheer 
numbers and diversity of the students, institutions, and educational objec- 
tives. To those sources of diversity can be added the thousands of courses 
being taught by hundreds of thousands of faculty members, each with con- 
siderable autonomy. In those circumstances, the failure to find evidence of 
results that can be attributed generally to our large, complex, decentral- 
ized higher education system as a whole is understandable. 
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A different kind of inquiry might examine how the coherent systematic 
structure that is apparent in American higher education despite its diver- 
sity has been established and is maintained. How are courses— -say in 
American history to 1860, or the first course in organic chemistry ,< or 19th 
century English literature— >-in widely different institutions, being taught 
to students who vary enormously in prior learning and academic interest,/ 
kept sufficiently similar that students can transfer from one institution to 
another in the same major field without starting over entirely? How can 
graduate students from widely different undergraduate ^colleges experience 
similar levels of success in the same graduate pjrtfgrams? How can .employers 
know what to expect when they require a college degree for employment 
without specifying which college must have awarded the degree? This kind of 
question will be addressed in this paper. A second kind of question to be 
addressed includes how the mystery around the residue of a college education 
or the mythology of what should be taught and how and when, might be reduced 
or how information might be provided on the quality of science education, or 
on what it is English departments do. 

The central role of the course in student learning 

Veysey (1973) has called the course the most durable element in 
American higher education. It is the unit in which the content to be 
learned is organized and through which instruction occurs. Even the learn- 
ing activities of students outside class are guided by the organization of 
the course. While much of the attention of curriculum committees, educa- 
tional policy committees, and other faculty groups is given to the ways 
courses may be strung together to constitute a worthwhile education, the 
actual instruction and learning are products of each individual course. 



Courses successfully completed determine a student ! s progression through 
college, from lower to upper division to graduation, from introd ictory to 
advanced material, from prescribed to elective content. Tl^ey provide 
structure to the student's major field of study. They determine how the 
student's general education pr breadth of learning is defined. The courses 
taken constitute the substance of an undergraduate education. 

Within a course, each faculty member determines the nature of the 
material to be taught — the content to be covered, its scope, depth, and 
detail; the way it is to be presented to the students and the activities 
in which they will be asked to engage; and what will be required of the 
students to demonstrate to the professor that they have indeed acquired a 
reasonable grasp of the course material. On large campuses, where several 
sections of the same class may be taught by different persons, the content, 
text, instructional procedures, and final examination may be common to all 
the sections. These courses tend to be introductory or basic lower-division 
courses taught by junior faculty or teaching assistants, and they often have 
their character determined by a senior faculty, member who directs the course 
according to his or her perceptions of what is important. 

Even courses that might be expected to be fairly standard across the 
country show significant variations. An example is an elementary course 
invariably found in electrical engineering programs usually titled something 
like Basic Electric Circuits. The course commonly covers "basic concepts of 
current flow," or "basic circuit theorems," starting with circuit analysis 
of direct-current circuits and ending with an introduction to alternating 
currents and sinusoidal wave forms, perhaps with a beginning on three-phase 
circuits. The elements of this course are in every electrical engineering 



program, varying only in their placement, such as whether three-phase 
circuits are introduced at the end of the basic courss or are held off for 
a 'second course. Yet this well-established course, with content that is 
widely agreed upon, is taught differently by different professors/ 

One faculty ©ember, when interviewed about how he approached that 
course, described hife objectiyes as getting his students to understand the 
basic concepts, equations, and problem-solving methods so they would be able 
to move successfully into the more advanced course. His procedure wa6 to 
introduce new material in a lecture, illustrate how to solve related prob- 
lems in circuit analysis, assign weekly sets of problems to be worked 
outside class, and review the solution of those problems in the first cla66 
session the following week. Another faculty meriber teaching the 6ame course 
at a similar university stated his primary objective to be to help hi6 
students learn how to think. The course content was secondary to the 
problem-solving process. His students, he said, would forget the specifics 
of the course within a few months of "completing it, remembering only what 
was reviewed and relearned in their following courses. But the processes 
of thinking-*-of knowing how to attack a problem, of applying equations in 
different types of problems, of verifying results — were likely to per6i6t 
and were the kinds of learning he wanted his students to have acquired by 
the end of the course. %is methods were not wholly different from those of 
the first professor, Although their goals and emphases varied. He also 
spent class time illustrating how to solve problems in electrical circuits, 
but more time was spent on fewer problems, with more emphasis on process 
than content. Basic concepts such a6 resistance and capacitance were 
introduced almost incidentally as systematic variations in a basic problem 
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as much as physical elements of an electric cir.cdit. In cpnt'rast, they were 
the heart of the course as taught by the first professor. 

Both classes were successful, at least to the extent that students who 
completed them had reasonable success in mof& advanced courses. > Beyond 
that, neither professor knew what the impact of his course had been — whether 
more than definitions of concepts and how to apply the appropriate equations 
to solve problems of circuit analysis had been or could have been learned, 
or whether their students were successful in more advanced courses only 
through extensivq relearning of the -material, or whether too much effort had 
been spent on some less consequential parts of the course at the expense of 
more usefu} learning. Both professors knew how well their classes performed 
on course examinations; both could compare their current classes informally 
with former classes; both believed, they were successful in teaching their * 
students what was most important in that course. But neither knew how 
well his students compared with students from similar classes elsewhere 
in the facility with which they handled more advanced courses or in the 
general usefulness of the uttfierstandiiigs or intellectual insights acquired 
in that course. 

Two issues are suggested by the above illustration. One is how 
such variations in faculty objectives and expectations come about. In 
contrast to the highly circumscribed course just described, faculty goals 
and expectations are substantially greater in such courses as Shakespeare V 
later plays, or the natural history of vertebrates, or introduction to 
philosophy. The upper-division courses in professional fields such as 
engineering and in the'physical sciences can also be expected to vary more 
widely than the lower-division courses as the basic groundwork in those 
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fields ±6 completed. These variations in" faculty perspective almost cer- 
tainly produce variations in the sco^e and depth of student learning. At 
present^, we have no Way tq exmaine those presumed variations iri^effect6* 

;, " 5 ' ■• ' < ' * 

The second issue illustrated by the course in electric circuits is what 
effects variations in faculty goals arid expectation^ may have on the broad 
aspects of student learning-— the aspects that persist after the details of 
the course have been forgotten or -that color the students 1 ' approaches to 
other courses or other intellectual activities. How are the purposes of the 
college oj university furthered by the activities* of faculty and students in 
their various courses? If we assume, as we must, that college courses have 
desirable effects on the students who take them, can we also assume that any 
reasonable approach to a particular course is as effective a6 any other? J 
the kaleidoscopic array of available courses from which students choose 
about 40 to constitute their undergraduate education,* how do variations in 
faculry goals and expectations within individual courses affect the educa- 
tional coherence of any student's chosen 40? 

No evidence can be found that the observable variability in courses and 
educational programs is" undesirable. A case might be made that programs are 
not variable enough, that the faculty members in some fields are too homogeir- 
eous in their backgrounds and present perspectives to provide as much diver- 
sity as is needed. Hodgkinson (1971) raised that possibility. Whatever its 
merits may be, the present system for establishing what i6 to be taught .and 

the levels of student accomplishment to be sought i6 not well understood, 

a- 

and its results are poorly documented. , 

If courses constitute the substance of an undergraduate education, and 
if faculty members haVe, great individual latitude in determining the content, 



instructional procedures, and standards for evaluating learning within their 
courses, the acKievements-^nd deficiencies of American higher education will 
be observed roos* directly in the course objectives set by individual faculty 
members and in the students' achievement of those objectives. The" Carnegie 

Commission (}973) -was discouraged from assessing the collective accomplish- 

■ * 

ments of American higher ^education by .the great diversity in institutions, 
programs, courses, and standards of achievement. Yet diversity in itself 
does not prevent the assessment of performance; it only tequires that the 
at »ssment reflect that diversity. 

The central role of the course in educational programs and the high 
degree of autonomy faculty members exercise in organizing and teaching their 
courses suggest that the v views of faculty members— -their perceptions. of 
educational purposes, of subject matter structure and importance, and their 
expectations for student learning— are major determinants of educational 
success. Despite diversity and local autonomy. in American higher education, 
it shows a substantial degree of coherence. 0 Students and faculty move with 
reasonable ease from institution to institution. The autonomy of individ- 
ual faculty members is countered by a complex network of .influences that, * 
without central direction, impose some order on the higher education system. 

* ■ 

Th v e remainder of this paper will describe, firsx, the nature of that faculty 
network and its constraining influence on faculty teaching and on educa- 
tional -content and standards. A second section will disciiss how *;he effects 

1 

of faculty views and expectations on student learning, as constrained by the 
existing system of influences, are and' might be assessed. Finally, proce- 
dures will be described that would extend -our knowledge of the educational 
accomplishments of the higher education system while allowing for its 
diversity. » 



Influences on faculty performances 

A^numbef of forces csri be identified that operate on faculty members 
to influence tbeir course planning , ceaching procedures, -and expectations 

for student learning. The following list is incomplete and heterogeneous 

» * 

in degree of importance^ but it provides a point of departure.- 

1. Prior educational and work experience; background 
characteristics. ' ' •' / 

. ' { 

2. Textbooks. 

\ • 

3. Direct contact Vith other faculty qrembere (e.g., * 
curriculum committees, departmental discussions, 

informal contact). i 

4. Professional literature. 

5. Professional associations; learned societies. 

6. The accreditation process. 

$ 

Faculty background . Faculty ©embers 1 initial efforts at course plan- 
ning and teaching are probably modeled closely after the courses they took 
as undergraduates or that they taught as graduate assistants tinder a senior 
faculty member. In their first: courses they may bfe given a course, outline 
or J syllabus from the course as it had been taught or, more likely, simply 
told what text had been used; Their organization and presentation of the 
course reflect their own experiences and their usually naive notions, 
despite their not tod distant experiences as undergraduates, of what a class 
of undergraduates can be expected to learn. Typically , - their early -expec- 
tations are far too high, based on an image ot undergraduat els that glives 
them the qualities of eager, interested graduate students, despite the 
reality of their great variability in. interest and capability. Since 
faculty members were among the most capable undergraduates, their own 
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experiences as a student do not give them an accurate picture of their 
current students* With experience, faculty members become more modest in 

their expectations and probably more successful in helping their students 

i 

learn. The initial teaching procedures and expectations . for student perfor- 
mance, though, flow from the faculty members 9 own prior experience as 

students and teaching assistants. Whether that experience, often in e a major 

. ■ \ 

research university, is appropriate for the less\ rigorous curricula and 

generally less competent students of the more representative colleges and 

universities, in which most faculty member^ teach, has been questioned (Doty 

& Zinberg, 1973; Dunham, 1969). 

One of the consequences of the leveling off of enrollments and the 
decline in financial support for higher education during the last few years 
has been a growing concern for the evaluation of faculty members as teacher's 
rather than primarily as scholars and researchers. Paralleling that trend 
has been a growth in programs of systematic attention to the improvement of 
teaching. As these programs mature, the influence on 6tudent learning of 
faculty members' prior experiences as students and their impressionistic 
views of the teaching and learning processes may decline. Yet the obser- 
vation of any such change may itself by impressionistic. 

While their own experiences as students and teaching assistants exer- 
cise a direct influence on faculty members 1 expectations for their students, 
other background factors— age, sex, social origins, and academic field, for 
example — have an indirect influence. Ladd and Lipset (1975), reporting on 
the Carnegie Commission's extensive faculty and student survey*, pointed out 
the strong relationships between faculty members' academic field and sex, 
race, religion, and social origin. Faculty members in law and medicine, for 
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example, were generally from families of comparatively high educational and 
occupational levels. Faculty in the undergraduate professional schools — 
education, engineering, business, and agriculture—came from more modest 
social backgrounds. The arts and sciences faculty members were somewhere 
between the other groups. 

■ \ 

One of Ladd and Lipset's major points, indicated in their title, 
The Divided Academy, , was the pervasiveness of the differences in points of 
view across academic fields, which followed fairly closely the differences 

a in social origins • In academic affairs, as well as in politics, faculty in 
the social sciences and humanities adopted more. liberal positions than did 
faculty in the physical sciences, who in turn, were more liberal than those 

s in the applied fields/ That order from liberalism to conservatism was 
reflected in the percentages of faculty members who would give students a 

m 

greater role in determining course content and procedures, who would liberal- 
ize grading practices, and who would relax academic standards to accommodate 
more minority students. The differences in the social and political values 
of faculty members thus were related to their educational views and presum- 
ably to their activities in class and their expectations for students. Yet 
Wilson, et al. (1975) found faculty values unrelated to student or colleague 
judgments of teaching performance. 

Similar differences across academic fields have been seen in ^iews 
jaore directly related to teaching and learning. Faculty members in the 
" natural sciences, in contrast to those in the social sciences and human- 

ities, gave more attention Hp their students 1 problem-solving performance 
and less to their verbal ability; were more concerned with the application 
of learning to new problems and less concerned with the students 1 personal 

ERIC ■ 14 . 



qualities (Warren, 1972), Faculty ©embers in the social sciences and the 
humanities focused more narrowly on the specific tasks of a course (Riesman, 
Gusfield & Gamson, 1970) compared^ with those in the natural sicences., 
gave more attention to their students 1 personal development, vanted a more 
personal student-faculty relationship, and were more inclined to trust 
their students 1 ability to direct their own studies (Riesman, Gusfield, & 
Gamson, 1970; Wilson, et al. , 1975), 

These differences in faculty members 1 orientation toward academic 
tasks have been frequently observed in one form or another (Bruner, 1960; 
Hirst, 1974; Parsons & Piatt, 1973), Whether they are determined by the 
characteristics of the field, the nature of the persons attracted to the 
field, or most likely, both, is in one sense immaterial. The social and 
academic backgrounds of faculty members and their preferred approaches to 
instruction are both strongly associated with their academic field. Expec- 
tations for student learning are more clearly defined, more specific, more 
closely tied to predetermined standards, and less adaptable to student 
variations in preference and ability among faculty in the physical sciences 
and applied fields, such as engineering and business, than among those in 
the social sciences and humanities. 

The preferences of students parallel those distinctions among fields 
of study, which may reflect a realistic and unsurprising confluence of the 
characteristics of subject-matter fields .and of the faculty members and 
students attracted to them. Yet that comfortable agreement in the perspec- 
tives of the participants in a field may preclude the asking of critical 
questions about the effectiveness of current teaching procedures and about 
the levels of achievement to which current procedures lead. The physical 
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Whether textbooks dominate the selection and organization of course 
content or faculty members adapt them to their own purposes — as derived 
from personal predilections, the curricular structure established by 
their department, and the capabilities and interests of their students — 
depends on the individual faculty member. Their decisions probably change 
with their own teaching experience and with the level and nature of the 
course. A course in a well-defined sequence for majors in the field is more 
likely to be tied closely to a textbook than is one intended for nonmajors 
or one that is optional for majors. 

Textbooks have done more than help faculty members organize their 
courses. They have also had a national impact on fields of study as to 
content, organization, and teaching procedures (Carnegie Foundation, 
1977). One text, for example, "profoundly changed lower-level biology 
teaching" (Doty & Zinb€^g, 1973). Three major texts between 1946 and 1953 
turned around the teaching of history, moving it from an overemphasis on 
sequences of events to deeper examinations of historical methods of under- 
standing and to the values that are revealed by and shape historical study. 
(Ward, 1981). 

The relationship between textbooks and faculty members operates in 
both directions. Most textbooks are written by faculty members. They 
serve, then, as a vehicle for communication among faculty members, re- 
flecting to some extent the collective facility experience with students 
in their courses as well as developments in the field. The influence of 
faculty experiences on textbook writing is biased, however, by the dis- 
proportionate numbers of textbook authors from a relatively few major 
universities (Bungum, 1980). Eaculty experiences in less rthowned 
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A second kind of formal faculty contact that is growing in frequency 
is provided by organised programs of faculty development (Centra, 1977; 
Lindquist, 1981). These tend to be more often built around teaching proce- 
dures and student satisfaction than around course content or student learn- 
ing. A faculty member to whom students give poor ratings as a teacher, for 
example, may learn, through an organized program for faculty improvement, 
that he or she expects prior learning that many students don't have, pre- 
sents the content without enough structure for the students to absorb it 
readily, or is unclear about his or her expectations. 

The organized professional contacts faculty members have outside the 
college or university, largely in the activities of professional associa- 
tions, are labeled semi -formal because their relevance to classroom in- 
struction is often incidental to their formal purpose. They may occur 
in groups organized around a discipline, a specialized area within a 
discipline, a broad issue such as values in higher education, a narrower 
issue such as the teaching of freshman composition, an institutional 
type, a curricular form such as general education or interdisciplinary 
studies, or they may be ad hoc groups concerned with almost any educational 
topic. These groups, organized for a .different purpose, may affect the 
way a faculty member will present a course, the kinds of learning expected 
of the students, or the level of accomplishment set as a course goal. 

Finally, the informal relationships faculty members have with 
colleagu.es in the same or different departments influence their course 
presentations, their expectations for students, and the wayfe they evaluate* 
student learning. Despite the importance of scholarly publication and 
research, to faculty members in many contexts, the great majority consider 
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teaching to be their primary responsibility (Trow/ 1977; Wilson, et al. , 
1975) and can be expected to exchange ideas and procedures related to v 
teaching with their colleagues. As faculty members grow in experience, 
their contacts with other faculty members as well as their own experiences 
might be expected to bring about a gradual change in perspective. Among 
slightly more than 300 faculty members in 15 California colleges and 
universities, those over 50 years of age were more likely than younger 
faculty to be concerned with the general intellectual growth of their 
students and to v*Iue strict academic standards. Those under 30 were 
least concerned with students' general intellectual growth, valuing the 
fixed structure and knowledge of the discipline more highly, and were 
most likely to be flexible in their standards for student performance 
(Warren, 1972). While Gaff and Wilson (1971) reported evidence consistent 
with those findings, Wilson et al. (1975) found no relationship between age 
and student or faculty nominations as effective teachers. The effective 
teachers were those who were most involved with the students in education- 
ally relevant ways, yhich might be expected to parallel the older faculty 
me~ T 2rs' concern for general intellectual growth reported by Warren (1972). 
High social involvement with students was not associated with student or 
faculty judgments of effectiveness as a teacher (Wilson et al* , 1975)» 

While faculty members exercise" great autonomy in their classes, 
their organization of their classes and their «pectationE for student 
learning must be moderated to some extent by their contacts with other 
faculty members. Yet only indirect and mixed evidence of that influence 
is available. 
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Learned societies and professional Associations . The learned 
societies— associations of faculty members based in the disciplines, such as 
the American Historical Association and the Modern Language Association — . 
have a somewhat -longer history than the professional associations; such as 
the American Dietetic Association, which are concerned with relations 
between their profession and society at large to a greater extent than the 
learned societies (Cheit, 197£). Some associations — the American Chemical 
Society and the American Psychological Association, for example—are members 
of both categories* Some associations of faculty members, such as the 
American Society for Engineering Education, have been formed to give their 
exclusive attention to educational Issues in a particular field. Other 
associations, such as the American Association for Higher Education and the 
Association of General and Liberal Studies, cut across disciplines. The 
various types of associations and the distinctions among them are not 
important to the purposes of this paper. The point that is of interest is 
that most faculty members have several professional associations with which 
they may affiliate, and many of the associations play a direct role in 
helping to determine the general content of courses that are offered to 
undergraduates. Their influence on the specific content is less certain. 

An informal survey of learned societies and professional associations 
by the Carnegie Council (Carnegie Foundation, 1977) indicated mote interest 
in graduate than in undergraduate studies and more in how the field is 
taught than in what is taught. Host associations resist specifying content 
for undergraduate curricula for fear of limiting academic freedom and 
stifling faculty ^^ativity. Yet some associations, after study by com- 
mittees composed largely if not entirely of faculty members, have published 
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recommendations that have heavily influenced undergraduate curricula. 
Griffith (1981) described a 1955 report by the American Society for En- 
gineering Education that had a major Influence in shifting engineering 
curricula toward greater scientific depth. A panel of mathematicians from 
education and industry under the auspices of the National Research Council 
(1968) reported on the topics that mathematicians in industry thought were 
the most serious deficiencies in their "undergraduate programs. By 1975, 
those topics were well represented in mathematics curricula (Warren, 1977). 

From time to time, apparently, the undergraduate curriculum in some 
particular field lags behind developments outside higher education in 
that field. At those times the prqfessional associations or learned 
societies carry out curriculum studies that lead to a restructuring of 
the undergraduate cutrfculum in that field. The committees that carry 
out these curricular reexaminations ■ are primarily college and university 
faculty members who bring to their deliberations some understanding of 
the changes in the field and the qualities of the students to be educated. 

The associations tend to address four kinds of issues when dealing 
with the undergraduate curriculum. First, they identify broad and often 
vague educational goals such- as "to improve our understanding of the 
world in which we live" (Dawson, 1981). Second, they recommend preferred 
sequences or patterns of courses, frequently in the lower division, as in 
the American Chemical Society's recommendation that physical and organic 
chemistry precede the course in instrumental analysis (Committee on Pro- 
fessional Training, 1978). Third, they develop and disseminate teaching 
materials (Carnegie Foundation, 1977). Fourth, they provide training 
programs that present some mix of teaching procedures and the organization 
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of content (Daw* on,* 1981). The direct Influence these activities have on 

the actual conduct of college classes is, difficult to judge. The impact of 

\ 

the first two is questionable despite the occasional influence of a report 
like that of the American Society for Engineering Education (Griffith, 1981) 
or the -National Research Council (1968). The third activity, the developr 
ment of teaching materials, probably has an appreciable effect on Intro- \ 
ductory courses with large enrollments and multiple instructors. The fourth 
probably influences classroom procedures more than content. 

Clearly, the learned societies and professional associations provide 
a vehicle in their education committees that brings faculty members from 
varied institutions toegther, sometimes with extracollegiate representatives 
of the field, to reexamine and revise undergraduate curricula. -The delib- 
erations of those committees reflect the teaching experiences of the faculty 
members involved, and their reports influence, occasionally in dramatic 
wayjs, the teaching of courses in the relevant fields. The comparatively 
high level of generality at which the associations operate, though, suggests 
that their impact on education may be strongest in the informal exchange of 
ideas about things to do in the classroom among the participants on com- v 
mittees and in professional meetings. 

Professional literature . In almost every academic field a journal 
can be found devoted primarily to college teaching in that field, and some 
fields, such as English and physics, have several. Almost all the issues 

faced by college faculty memberB are addressed. Articles can be found on 

A 

(1) presenting particular course content, (2) teaching general intellectual 
skills, such as problem solving, rather than specific content, (3) the use 
of various pedagogical devices such as computer-assisted instruction, small 
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edited a book that takes a new and fairly comprehensive look at higher 
education and that gives a major section to the academic disciplines 
and their as ociated curricula. Even the chapters on the disciplines, s 
though, and one on instructional methods give almost no information on the 
substance of student learning. The books produced by the Carnegie Com- 
mission and the Carnegie Council (1980) dominate the recent literature on 
higher ..education and are an impressive and comprehensive accomplishment. 
They also, though, show the same gap in information and understanding that 
appears in* the journals. 

Cue book., now ten years old; addressed directly the question asked 
above on how faculty members might evaluate their own teaching effective- 
ness. Mil tor (1972), drawing his information from published and unpublished 
studies ^as well as personal experience, described faculty members as 
generally confident that their teaching was quite good and uninterested in 
iooking closely or critically at its effects. The studies Milton reviewed 
reinforced his own observations that, although students leave college having 
learnfed mich they did not know at entrance, how that learning is most 
readily brought about is still largely a mystery. One teaching procedure 
see ms no better or worse than any other, and at times students learn as well 
without an instructor as with one. More. recent reviews have provided 
nothing to change Milton's conclusions (Centra, 1978; Levinson-Rose & 
Menges, 1981; McMillan, 1975). 

The professional literature, books as well as journals, are an im- 
pressive and comprehensive source of a variety of kinds of information that 
- can help faculty members define their teaching objectives, organize their 
courses, keep them current, relate them to general curricular goals, identify 



useful teaching* techniques and materials, and improve their teaching 
capabilities. The effects of any of these activities on student learning, 
though, is still the mystery that Bowen (1979) noted. While our graduates 
are, on the whole, competent and reasonably knowledgeable in their fields, 
we know little' of the scope or detail of that competence or of how it was 
acquired. ... 

Accreditation .' The accreditation process is a well-established proce- 
dure through which institutions and professions exercise some control over 
the programs of institutions at large. The various accreditation commis- 
sions- — those of the regional aspociations which accredit colleges and 
universities a§ total institutions, and those of the specialised associ- 
ations which accredit programs of a single ^profession within a university 
or specialized school — are composed largely of academic administrators and a 
few faculty members. They establish "Standards of institutional organization 

and procedures that reflect the collective perceptions of what constitutes 

' i' 

good practice among administrative and faculty leaders in currently accred- 

~% 

ited institutions and programs. Their authority depends on the acceptance 
by most of the existing institutions and the general public of the soundness 
of their standards and procedures. 

The accrediting associations f^ce the sarie dilemma as the professional 
associations in attempting to assure minimum levels of educational quality 
while allowing for great insitutional diversity and autonomy. In fact, many 
- of the specialized accrediting agencies are either offshoots of or are still 
affiliates of professional associations. The American Chemical Society, for 
example, provides an approval program similar to accreditation for undergrad- 
uate chemistry departments through its: Committee on Professional Training. 
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Another dilemma the accrediting associations continue to struggle with 
is whether their primary purpose is to define and maintain minimum standards 
of educational quality or to promote continued improvement in quality 
whatever an institution's or program's existing level of quality may be. 
Both dilemmas force the standards each accrediting agency publishes to be 
general enough for broad application in any situation. For example, vir- 
tually every accrediting agency requires that the faculty of an institution 
or program be qualified in the areas in which they teach, but the dimensions 
and standards of quality are left undefined. The actual decisions about 
faculty quality, and its implicit definition, are left to the current 
members of the accrediting commission, who act on information provided by a 
visiting team drawn from faculty members and administrators of similar 
institutions. Bow severe the constraints should be on an institution's 
decisions about the qualifications of its faculty, and whether a program 
with a faculty capable of providing a minimally acceptable education should 
be denied accreditation if the faculty is not capable of providing much 
educational improvement, illustrate the two dilemmas. 

Both dil emma s are resolved partly through direct observation of an 
institution or program by a visiting team of experienced faculty and 
administrators who exercise their professional judgment within the broad 
guidelines of the published standards. Whether a faculty "is fudged accep- 
tably qualified or not will then depend on other aspects of the institution 
such its educational objectives, teaching procedures, administrative 
strengths, and student characteristics. The first dilemma— between assur- 
ance of quality and acceptance of institutional diversity— is resolved to 
some extent by giving each institution or program the responsibility of 
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describing, through a self-study, the ways it satisfies the agency's stair 
dards. 

. These two major accreditation functions— the visiting team and the 
self-study— act as vehicles for the exchange of perspectives on educational 
practice and performance. A" program or institution, faced with the task of 
providing evidence that will convince an external group of experts that it 
is successfully educating its students, must find some way to document its 
\ acuity's beliefs that their courses are indeed soundly conceived and 
effectively taught. That exercise in self-evaluation can be informative, 
sobering, and a prod toward faculty improvement. The visiting experts then, 
in reviewing. the self-study with the faculty, make informal comments and 
suggestions, drawn from experiences at their own institutions and from 
similar* visits to other institutions. Visiting team members will often 
leave with new ideas and perceptions picked up from the institution visited 
The accrediting process thus serves as an information exchange, informing 
faculty members of effective practices at other institutions, as well as an 
incentive toward self-examination. 

The collective wisdom of experienced faculty members is a major 
"'educational resource, and the accrediting process systematically spreads 
that wisdom. let the selfr-study or visiting team that gives attention to 
what students have' learned is rare, if any exists at all (Peterson, 1979; 
Warren, 1980). Howard Bowen (1979) argued forcefully for the evaluation of 
student intellectual and personal growth in assessing quality in education 
while pointing out the inadequacy of our present reliance on intuitive 
judgments, however sensitive and perceptive they may be. 
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^ [W]e might gradually through trial and error develop sound 

methods which would supplement our intuitive judgments and 
serve as a corrective for the wishful thinking and the 
empty rhetoric in which we all indulge (Bowen, 1979, p. 28). 

«-4 * * " 

The network of faculty influences 

The six sources described above on which faculty members draw in 
planning and teaching their courses are clearly interrelated. All six 

jlraw Jtheir^PKii.fQrce-J^ lt toward _ 

further faculty development. None exercises any formal authority on 
another except as the accrediting agencies may put the pressure of codified 
standards on faculty in programs seeking professional* and public acceptance. 

Faculty members enter their teaching careers with attitudes toward 
teaching and views of the important content of their disciplines that were 
shaped by their experiences with their own teachers. In their early teach- 
ing years, the structure they give their courses, the teaching techniques 
they use, and their expectations for their students are probably influenced 
most heavily by their own experiences as students, by the textbooks they 
use, and by other faculty in their own department. As they become more 
experienced they are more likely "to teach advanced courses, to rely less on. 
the organization provided by a text, and to draw from broader faculty con- 
tacts in professional associations and perhaps in accreditation activities. 
For some faculty members, the literature on college teaching in their fields 
will exercise some influence. But while the existence of these influences 
on college teaching is undeniable, the extent and forcefulness of. their 
impact, their relative importance, and the extent to which they reinforce or 
interfere with each other are total mysteries. 

' " • "V. 
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Each of the six influences described is based almost entirely on the 
views of other faculty members. Even academic vice presidents and deans are 
limited in their points of contact with the network of influences, entering 
it primarily in their establishment of faculty committees and in the accred- 
iting process. And in both these activities they operate through faculty 
members. All faculty members are connected to this network at least through 
their experiences as graduate students and their contacts with other ©embers 
of their department. Most probably they participate in four or more sources 



of influence, which -suggests that information probably flows quite effi- 
ciently through the network. Information that starts with only a few people 
in an accrediting visit or in an ad hoc committee representing some faculty 
interest group will be taken to other faculty members, move to other pro- 
fessional groups, appear in, prof essional articles, be picked up by graduate 
students and become part of their courses when they start to teach, and will 

eventually become incorporated into textbooks. 

I 

|The winnowing that occurs as new ideas, instructional devices,' and 
curricular structures are moved through the network probably accounts to 
a laxjge extent for the consistency that can be observed across the country 
in academic content and teaching procedures, at least within major fields of 
study!. Despite the great diversity and institutional autonomy of American 
higher education, to & large extent a bachelor's degree in a given field of 
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stud^l in a state university in Idaho arid one from ;the. same field at a state 
university in Tennessee represent fairly simila/educational experiences. 

ConaeiUus is broad while individual institutions and faculty members are 

/ 

free t;o experiment. The schisms that ocqtir within disciplines, as in 
Marxist challenges to the dominant view of American history or in a 
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reformulation of the functions of literary criticism, do not invalidate the 
contention that curricular consensus is widely held. When alternative views 
are in dispute, they tend to be clearly distinguished, and the broad areas 
of curricular agreement are unaffected. 

Hodgkinson (1971) commented with some concern on the growing homo*" 
geneity he found among colleges and universities in the U.S. despite their 

autonomy . Ea ch institution r independently -of-^wny "ether, seemed to want - to 

be as much as possible like the major research universities. He would have 
preferred to see institutions cherish their differences, building their own 
vefi ions of excellence. To aid a process of differential development, he 
proposed establishing a formal information network, probably managed by the 
professional associations and learned societies, that would gather and 
circulate a wide range of information on statistical trends, curricular 
changes, instructional procedures, and other kinds of information that would 
give faculty members, administrators, students, and any other interested 
group a context in which to observe institutional performance. The conten- 

0 

tion here is that much of the formal network Hodgkinson proposes already 
exists informally in the range of faculty influences described earlier. 
Whether those formal and informal processes would be more useful if they 
were formally integrated and centrally managed requires information not now 
available. Their current effects and the ways they are accomplished need 
more explicit examination. 

Despite apparent plausibility, much in the preceding paragraphs is 
speculative. We know that students graduating from different colleges 
or universities have spent comparable periods of time in classes that were . 
probably similarly structured. We know that graduates in the same field of 
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study have a reasonably common core of knowledge on which they have expanded 
in one of perhaps four to ten alternative patterns (Warren, 1975, 1976(b), 
1977). We don't know, however, what faculty members, who participated in a 
complex but informal information network, actually presented to their 
students-— what range of content and procedures and what levels of expecta- 
tion the students encountered, or what elements of learning were reasonably 
common. Little, if any, information exists on what the collective behavior 
of the faculty members produced in student learning. We continue to operate, 
as Bowen pointed out, on intuitive judgments, wishful thinking, and empty 
rhetoric. 

Intuition and wishful thinking both lead to the belief that the American 
higher education curriculum, as it applies to the education in •some depth of 
students in their major fields, is a successful blend of consensus and 
diversity. Achievement of the broad goals of higher education intended 
to be the purpose of general education is more questionable (Carnegie 
Foundation, 1977; Warren, 1975). Both beliefs—the success of specialized 
education and the failure of general education— are capable of documentation 
if the extent and dimensions of the diversity that exists are defined, if 
the areas and degree of consensus within that diversity are identified, and 
if student learning is evaluated in ways that take account of diversity and 
consensus. None of these contingencies is out of reach. 

Evidence of student learning 

Defensible evidence of the effects of college teaching on student 
learning is difficult to find, as has been repeatedly pointed out. The most 
widely used standardized tests of college achievement, the Advanced Tests of 
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the Graduate Record Examinations, permit comparisons of the achievement of 
graduate school applicants from year to year, and institutions may compare 
the performance of their graduates who take the GRE Advanced Tests (as 
opposed to the \ il, Quantitative, and Analytical Aptitude Tests) against 
the performance of students nationally. Several characteristics of the GRE 
tests, however, limit the usefulness of those comparisons. 

Changes from year to year in the achievement of graduate school appli- 
cants are interesting but difficult to interpret, primarily because the 
nature of the students taking the different tests changes in unknown ways as 
different graduate fields of study shift in their attractiveness to students. 
From the mid-60' s to the mid-TO's, for example, Advanced Test scores of 
graduate school applicants in some of the physical sciences increased while, 
those in the social sciences declined and those in the humanities stayed 
about the same. The numbers of applicants taking the various tests in- 
creased dramatically in some fields,, such as biology and geology, while 
declining sharply in others, such as engineering and history. These changes 
in numbers of applicants, however, were not related to changes in mean test 
scores. Increases and declines in test scores occurred with changes in both 
directions in the numbers of persons taking the test. , 

Institutions examining changes in the Advanced Test tocores of their own 
graduates, or comparing their graduates with the most recent norms, face 
other interpretive difficulties. The most serious is the nature of the 
content covered by the Advanced Tests. Test content is determined by 
committees of faculty members who see that each^ test keeps pace with changes 
in its field while including content likely to have been encountered by most 
students majoring in that field. The tests range in numbers of items from 
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about 100 to 200, with most having at least 150 item*. That number of items 
can sample a wide range of content and provide an acceptable assessment of a 
student's grasp of a field in general. Even 200 items, though, is not 
enough to allow for differences In patterns of emphasis within a 3:ield. 

The Advanced Test in Political Science, for example, has 170 items, 
30 to 35 percent of them on the U.S. government end another 20 to .25 percent 
on comparative political systems. The rest of the items are spread roughly 
equally among international relations, political theory, history of polit- 
ical thought, and methodology. Students majoring in political science, 
though, often do not arrange their programs to match that balance of con- 
tent. Institutions differ in the various patterns of content their majors 
follow. Some political science departments stress quantitative methods 
while others do not. Transcripts- of graduates frcm 10 political science, 
departments showed seven clearly distinct patterns of, content, with addi- 
tional minor variations (warren, 1976b). One common pattern gave primary 
emphasis to political theory and secondary emphasis to international re- 
lations and economics. Another showed heavy concentrations in the federal 
government and the political process , with organizational theory, political 
theory, and economics playing secondary roles. Students who followed this 
second pattern -in their undergraduate studies would probably perform better 
than students from the first pattern on the political science Advanced Test, 
other things being equal. 

A test designed to represent the content that is most common to the 
departments in a given field across the country cannot adequately represent 
the learning of graduates from individual departments . The GRE Board 
recognizes this problem, as indicated in their bulletin describing the 
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Advanced History Test, 

The problem of content coverage in a single history test is 
complex. It is almost impossible to delimit the rield of 
history in area, in time, and in scope. Moreover, no common 
core of knowledge is required of all history majors in all 
colleges ( A Description of the Advanced History Test , 1979). "" 

The test is limited to United States and European history, excluding Asia 
and Latin America. It concentrates on political and diplomatic history, 
gives Secondary attention to economic history, and pu.ts social, Cultural, 
and intellectual history in minor roles. The questions on U.S. history _ m 
refer predominantly to the period after 1800, and the majority of those oft 
European history- follow the Industrial Revolution. History before the 
Middle Ages is entirely neglected. As with political science, the course 
patterns of history majors vary widely among themselves as well as 
departing from the pattern on the test. 

For the primary purpose of the GRE,, these acknowledged limitations 
are minor. The tests are to assist graduate schools in evaluating the 
preparation of their applicants. Most of the content of the history test 
will have been encountered by most history majors even when their emphases 
are elsewhere — on ancient history or Latin American history, for example. A 
history department that uses the GRE Advanced Test to evaluate its program, 
though, may find its particular strengths and many of its upper-division 
courses inadequately represented. Yet the GRE Advanced Tests provide 
virtually the only information based on student learning now available to 
departments for determining the quality of the education they' offer. 

The most recent evaluations of the accomplishments of colleges and 
universities in furthering student learning or development continued to 



neglect the substantive, course-related learning toward which the resources 
of higher education are overwhelmingly directed. Astin (1977) summarized 
data from a series of longitudinal studies involving a total of more than 
200,000 students in more than 300 colleges and universities. He reported 
changes in students 1 attitudes, interests, career plans, and extracurricular 
activities.. His indicators of academic accomplishment were grades, persis- 
tence in college, and academic honors. These measures distinguished the 
more successful from the less successful students. They gave no information 
on what the more successful students learned that distinguished them from 
those who were less successful, or on what students learned regardless of 
their level of success. That information would, of course, be extremely 
complex, varying with field of study, institution attended, and the educa- 
tional purposes; and expectations of the various faculty members who assigned 
the students' grades. Complex as it is, however, information on what 
students at every level of success have accomplished in college, as well as 
on the maximum levels of accomplishment reached by the most successful 
students, is essential to any clear assessment of educational quality. 

*» 

Bowen (1977) search^ for whatever evidence he could find of the 
* „ effects of higher education, \on society as a whole as well as on individual 
students. The components of individual student growth for which he found - 
evidence were classified, much like Astin 1 s, as cognitive or intellectual, 
affective or attitudinal and emotional , and practical. Drawing on the 
entire body of literature on higher education, rather thin on the variables 
of a gingle though comprehensive series of studies as Astin had, Bowen found 
some evidence ^of student growth in broad intellectual skills, substantive 
knowledge, dispositions toward intellectual and esthetic activities, and 
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intellectual tolerance,. The broad intellectual skills were measured by 
verbal and quantitative tests such as the GRE Aptitude Tests, and the 
substantive knowledge by' tests like the GRE Advanced Tests » As pointed out 
earlier, tests such as these are too heavily influenced by whatever is 
commonly taught in all types of colleges to be sensitive to the accomplish- 
ments of successful students in the more selective colleges or to the 
variations in accomplishment across the great variety of colleges that make 
up the American higher education system. The intellectual dispositions, 
while valued by many faculty members and indicative . of intellectual growth, 
represent only minimally the broad intellectual objectives of most faculty 
members. 

a 

While Sanford's 1962 volume brought to wide attention the inevitable 
interplay between higher education and the psychological development of 
students, Chickering's 1981 volume makes a direct argument for giving to , 
human development in its broadest sen^e the central, organizing role in 
higher education. "We propose that the values and aims of human development 
be taken as unifying purposes, as organizing frameworks for all institu- 
tional efforts. . (Chickering, 1981, p. 11). Faculty members generally 
support such a view. From 60 to 80 percent of the faculty members polled by 
the Carnegie Commission in 1969, depending on the type of institution in 
which they taught, agreed that undergraduate education would be improved if 
more attention were paid to the emotional growth of students (Trow, 1975). 
If "emotional growth" were replaced with a term like "broad intellectual, 
■oral, and personal development," the percentages of faculty members endor- 
sing that statement would almost certainly be overwhelming. 
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Two years after the Carnegie survey, about 300 faculty members repre- 
sentative of the faculties in 15 diverse California institutions were asked 
to locate their views of educational purposes on a dimension defined by the 
following sbmewhat disparate statements: 

. Colleges and universities have important responsibility in 
furthering students' psychological growth and personal 
maturity. 

Colleges and universities carry out their responsibilities 
best by limiting their concerns to the intellectual growth 
of students* 

Fifty percent of the faculty members placed their views closer to the first 
Statement than the second, and another 23 percent called the two statements 
equally supportable. Only 8 percent allied themselves definitely with the 
second statement even though it permits a broad interpretation of intellec- 
tual growth (Warren, 1972). 

r 

The concern of Chickering and the writers he brought together for all 
aspects of human development as explicit educational) objectives should find 
a receptive aucjjience. In fact, Astin and Bowen found more evidence of 
college effects in attributes like intellectual curiosity, tolerance, 
esthetic sensibility, and personal satisfaction than in some of the more 
explicit educational objectives of the disciplines, such as the ability - 
to solve problems. The e^dence for college effects on student growth in 
those aspects of development that are closely related to courp* and curric- 
ular objectives, excepting only the limited aspects of subject-matter 
knowledge represented by standardized achievement tests like the Graduate 
Record Examinations, £s, in Bowen' s terms, -spotty, partly obscure, and 
incompleted (Bowen, 1977, p. 97). 
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The primacy of the classroom 

The primary investments of colleges and universities are in their 
faculty, their courses, and in the classes through which the courses are 
taught. Whatever learning is mosrt valued by the administration and faculty 
must be carried out through faculty-directed courses. 7 Other important kinds 
of learning, intellectual as well as social and psychological, occur outside 
the classroom and outside the direction of faculty members. The total 
development of students that Chickering would ptjt at the core of higher 
education probably cannot be separated* from all aspects of higher educa- 
tion — classroom and laboratory, extracurricular, residential, and inter- 
personal. But the Accomplishments of higher education that are highest in 
importance in terms of student learning as opposed to research and public 
service must be brought about primarily in the classroom. If faculty, 
administrators, trustees, legislators, or other institutional constituencies 
place a high value on a particular form of student development, perhaps 
growth in interpersonal skills, it oust be addressed in the classroom if it 
is to be given attention appropriate to its importauce. The primary goals 
and accomplishments of higher education appear in what, faculty members and 
students give their attention to in the classroom. 

If that apparently obvious assumption can >e accepted, two useful 
inferences follow. First, the network of influences that shapes the organ- 
ization and presentation of courses by Individual faculty members — prior 
experiences, textbooks, professional associations, professional journals, 
faculty committees , accrediting teams, and informal contacts with col- 
leagues — provides an Important mechanism through which educational quality 
is shaped. The interconnections among those influences provide stability 
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and some uniformity, but also the potential for change, A shift in any one 
of the influences may riot have much effect." Complementary, shifts in several, 
of the Influences— as when a core of faculty members at several institutions 
devises a new approach to teaching some component of their discipline or a 
cross-disciplinary component of learning, and that approach is discussed in 
professional meetings, appears in journal, articles, and is developed in a 
new text — give ney departures from former practice a moderate probability of 
persisting and spreading. As faculty members and students get exposed to a 
new procedure, it may continue to spread,, with modifications^ or it may 
have a brief period of wide acceptance and then die back; or it may persist 
in some portion of the higher education community as an alternative approach 
to an area of learning, contributing to the diversity of the system. A 
heavy emphasis on quantitative methods in political science may be such a 
persistent alternative that fails to spread to the field as a whole. 

The second inference from an -assumption of the primacy of the class- 
room is that any comprehensive body of evidence of student learning, or of 
educational excellence, must be aggregated from evidence taken at the level 
'of individual courses. The diversity of courses from institution to insti- 
tution in the same field is great enough, despite the external influences to 
which faculty members are exposed, that standardized test scores, grade- 
point averages, persistence to a degree, and proportions of graduates 
admitted to graduate schools are all too coarse t provide sufficient 
information on the educational quality or a school or department. None of 
those indicators reflects the variety of accomplishments, 'understandings, 
appreciations, and insights that students may dray from the courses faculty 
■embers present. The aggregation of course-based information, however, can 
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define the components of learning that are common to more than one course 
and more than one institution, or even to the great majority of institutions, 
while identifying those kinds of learning that occur only in a small segment 
of the higher education community. Some forma of undergraduate learning in 
the physical sciences and mathematics, for example, may be found only at Cal 
Tech, MIT, Harvey Mudd College, Rice Institute, and perhaps five or six 
other institutions across the country. Other patterns of learning that are 
limited to comparatively sigall groups of institutions can almost certainly 
be found in all disciplines and in interdisciplinary courses, in advanced 
courses designed for majors in a field and in general education courses 
planned for nonmajors. 

Aggregating information from courses 

If most of the learning in higher education is organized in courses, 
the effects of higher education on students can usefully and accurately be 
described in terms of the aggregated effects of those courses. Aggregation 
in a simple form is well established in higher education. Units are added 
to units to determine a student's progress toward a degree; courses are 
added to courses to assure that each student follows an appropriate pattern 
of courses; grades are converted to numbers arid averaged for an indicator of 
overall student success. Lost in all these forms of aggregation is the 
substance of the learning. 

Several ways of aggregating indicators of educational success that 
retain the substance of what has been learned are feasible. The reason for 
their neglect nay be the almost exclusive concern with assessing the achieve- 
ment of individual students rather than that of selected groups of students, 
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such as those' completing different sequences of courses in a field. The 
level of aggregation can be selected to suit the purpose of the assessment, 
such as whether interest is on the achievement of all the history majors in 
an institution over the past five years or in the achievement of the history 
majors who have just completed a new sequence of courses in historical 
©ethod. The level of aggregation may also vary according to the generality 
of the content, from very specific knowledge about the Italian Renaissance, 
for instance, to a broad understanding of the interplay of politics and 
religion in 15th century Europe, and may involve general abilities like 
skill in historical inquiry. With higher levels of aggregation, some 
information is lost. The learning that is characteristic of all senior 
history majors in an institution, for example, will only be described at 
the expense of more detailed inf ormatipn about the effects of a particular 
sequence of history courses. 

An example of one form of aggregation across the graduates of a group 
of institutions in a single discipline is provided by an examination of 
the transcripts of five graduates in English at each of 11 institutions. 
The data were collected in the conduct of a larger study supported by the 
Fund for the Improvement of Postsecondary Education (Warren, 1975, 1976a, 
1976b, 1977).- 

The usual information on the educational accomplishments of an English* 
department consists of the number of its graduates, perhaps the proportion 
admitted to graduate school, and perhaps the mean scores on the GRE Advanced 
Literature in English Test. But English departments vary in their strengths, 
emphases, and faculty and student predilections. Students' programs within 
a department vary as well. A more revealing indicator of an English 
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department's accomplishments would therefore be a description of the pat- 
terns of courses completed by its graduates. {'Grades would not be informa- • 
tive since they refer to students' comparative achievement within the 
department, but mean scores on the GRE and proportions of graduate school 
applicants admitted to their first choice of graduate schools would add some 
supplementary information. 

The 55 transcripts selected randomly from^he English graduates of 11 
institutions provided a list of the English courses each graduate had 
completed. College catalogs provided descriptions of the content of each 
course. Four to eight elements of content from each course description 
were listed to augment each student's transcript providing more descriptive 
information than the course titles alone. The graduates, or more precisely, 
the augmented transcripts, were then grouped according to the similarities 
in the patterns of content that had been studied. 

The five transcripts from one moderate-sized state university showed 
more than the usual emphasis on drama, the writing of plays, novels, and 
short stories, and literary criticism. Comparatively brief treatments were 
given to English literature in the several periods from Chaucer to the 
present, and little time was spetit with American literature. Shakespeare 
was treated as part of the English Renaissance. In contrast, the trains 
cripts f jrom a small private college showed a methodical and comparatively 
detailed progression in the study of literature from the medieval period 
through the Victorian Age with brief excursions into modern European lit- 
erature, the analysis^of poetry, and modern grammar. The limited study of 
writing was wholly expository « 



A large public university showed more varied patterns of study that 
differed from both the first two. All five of its transcripts showed the 
study of Shakespeare, English literature in the Romantic and Victorian 
periods, the modern American novel, expository writing, and modem grammar. 
In addition, three of these graduates had studied the novel in England and 
America in some depth plus several modern genres, structural linguistics, 
transformational grammar, and semantics. The other two transcripts were 
more similar to those of several other institutions, showing extensive study 
of English, American, and world literature without notable departures into 
drama, expository or creative writing, or linguistics. 

The content common to all 55 transcripts studied was quite limited. 
All included the study of Shakespeare and the Romantic and Victorian periods 

in England. Most but not all included some modern American literature. 

«> 

Beyond these areas, little could be found that was common to more than small 
clusters of transcripts. If such an analysis were to be carried out on the 
transcripts of all the English graduates of a single institution over 
several years, the variability would probably not be as great, but clearly 
defined patterns would no doubt appear that would differ substantiall> xrom 
the similarly determined patterns at other institutions. Some of the 
distinctions would be expected and valued; others might be neither. What- 
ever value the faculty, department head, and dean might place on a descrip- „ 

ton at that level of detail of the patterns of learning the department's 

i 

graduates had taken with them, they would be in a sound position to evaluate 

what they were accomplishing as a department. 

\ 

Different levels of aggregation would serve other purposes. Several 
major components could be examined in more detail— Shakespeare, the Victorian 
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period, the mbdera American novel, for example— to determine the scope and 
depth with which each was being treated. At a more general level, the broad 
aspects of the major field might be combined with the components of studies 
outside the major, giving a picture of the areas of study common to most 
graduates and of the variety of ways students elaborated on that common 
core. What it is English departments do would be somewhat dearer. 

Information of the type illustrated above can be used to describe the 

* +4. 

similarities and differences in the studies college students undertake 
within a single department in a single institution or across all fields in 
large numbers of institutions. As the domain of Interest is broadened, the 
descriptions will necessarily become more diffuse and varied, but elements 
common to groups of institutions will appear, reflecting the influence of 
the faculty network that extends across institutions. 

Facility expectations for general learning 

the aggregation of course information from transcripts and catalogs 
can provide valuable information on the substance of the courses students 
encounter in moving through college. But course content is not the only 
kind of learning faculty members expect of their students, and some, even in 
practically-oriented fields like engineering, view course content as secon- 
dary to the development of more general intellectual capabilities— the 
ability to identify the critical elements in an idea, argument, or problem; 
to review a topic as a whole and gather and organize relevant information; 
to integrate the new information into an existing framework, or to use it to 
restructure the existing framework. Teaching objectives such as these are 
less often and less clearly articulated than the elements of content that 
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appear in course syllabi or catalog descriptions. Yet they too can be 
described and aggregated to show the educational objectives of a small 
group of faculty members in a single institution or the collective objec- 
tives of faculty members at large, revealing the areas of broad consensus 
and the nature of the differences. 

This kind of aggregation can be illustrated with information from 
faculty descriptions of outstanding students (Warren, 1972, 1976a). A 
random sample of 311 faculty members in 15 California colleges and univer- 
sities wrote a paragraph or two describing how one of their outstanding 
students differed from the ordinary students in one of their classes. Those 
descriptions produced a list of about 10P descriptive phrases, such as "sees 
several ways to solve difficult problems" and "is sensitive to the nuances 
of language," that were assumed to represent explicit kinds of intellectual 
performance valued by faculty members. Some of the phrases were oriented 
toward content, such as "knows the technical vocabulary. " Most, however, 
were more general, applicable to a wide range of content. 

Several years later, a new group of about 500 randomly selected faculty 
members at another group of 15 California colleges and universities judged 
the degree of similarity among the phrases. They judged, for example, that 
calling a student "perceptive" was very 'Similar to saying he or she "has 
insight." These two phrases and others like "differentiates the general from 
the particular, the important from the unimportant" and "alert for discrep- 
ancies, inconsistencies" were all judged to be mutually similar, forming a 
general quality of being critical or analytic. Other phrases clustered into 
concepts like inquiring, integrative, expressive, original, and simply 
academically competent in the sense of doing well on exams. 
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Fourteen' such clusters of descriptive phrases appeared, defining in 
broad but distinct terms the kinds of intellectual abilities faculty members 
want their students to bring to tear on the substantive knowledge around 
which courses are typically organized. Any complete assessment of educa- 
tional excellence would describe the areas of knowledge, understanding, or 
appreciation the graduates of a program had encountered and the kinds of 
intellectual skills they had been called on to exercise in dealing with 
that substantive content. The skills as well as the content will vary with 
fields of study, institutions, departments, and faculty members. That 
variation should be conscious and purposeful, though, with decisions to 
focus on one area to the neglect of another made deliberately and with 
knowledge of similar decisions and their consequences elsewhere. 

The faculty 1 judgments used to classify phrases into groups suggested 
one decision that may have been reached, implicitly without conscious delib- 
eration. A phrase that describes general intellectual competence — "capable, 
bright"— -was grouped with .the phrases that defined analytic, critical 
thinking. The integration or synthesizing of ideas was described separate- 
ly. One inference might be that many faculty members associate being 
capable and bright almost entirely with analytic processes, with little 
regard for integrative skills. Some support for this interpretation is 
found in the reports of faculty committees reviewing general education 
requirements , in which analysis-— philosophical, historical, literary, 
scientific, and other forms — appears more frequently than synthesis or in- 
tegration as an educational concern. The apparent emphasis on analytic at 
the expense of integrative thinking seems not to have been a deliberate 
decision, yet present curricula have that bias. 



A faculty-based assessment system 

The individual course, as presented by largely autonomous faculty 
members in their classes , is at the center of learning in American higher 
iwiucation. the constraints on faculty members that provide structure to the 
learning that occurs come from an informal network of influences that are 
lar'g^ly self-generated by the faculty they constrain. Individual courses 
are given similar content and purposes and produce comparable results, even 
though taught at widely separated institutions by faculty members who have 
no direct contact with each other, through that faculty network. A reversal 
of that process can serve to demonstrate the level of excelience reached by 
American higher education by aggregating the learning produced in individual 
courses at successively more comprehensive levels* 

The knowledge of content, understanding of general issues, and exercise 
of intellectual Skills that constitute undergraduate learning are almost 
wholly associated with individual faculty members and their courses. Yet 
among all those courses, individually planned and presented though jthey may 
be, are common elements of content, understanding, and skill that can be 
aggregated to describe learning more general than that associated with a 
single course. Many lower-division courses in American history, for example, 
study the conflicting political philosophies of kamilton and Jefferson and 
their partial integration .by Madison. Knowledge of that constitutional 
controversy could be expected in students completing those courses. Some 
understanding of the interplay of the political, social, and economic issues 
that influenced the framing of the constitution, and of the processes of 
negotiation and compromise that permitted their resolution, could be ex- 
pected in students completing those and other courses. The ability to 
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generalize usefully and with appropriate caution from historical events to 

• A 

current issues that are in some ways similar could be expected of still 
larger groups of students* Each of those increasingly general kinds of 

learning, and others, could be assessed in individual' courses, with the 

p \ 

results aggregated across the appropriate range of courses. \ 

The proper selection of a group of courses and assessment in them of 
the pertinent intellectual capabilities would provide evidence of educa- 
tional accomplishment at whatever level of aggregation was desired — several 
sections of the same course at one Institution, all the lower-division 
' courses in a single department at one institution, all of a given group of 
courses, say t^he American novel, at several neighboring institutions, all. 
the courses taken by the graduating seniors in a given field in a statewide 
univesity system. Assessment would have to be in the individual courses, 
usirig common measures across the courses to be aggregated. If that* assess- 
ment were made an integral part of the course examinations it would require 
comparatively little additional cost, primarily that associated with the 
development and acceptance of common examination questions for the pertinent 
courses. At the highest level of the higher education system as a whole, 
results from such assessments could be used to describe the kinds of learn- 
ing produced in various fields of study, the kinds attributed to general 
education or curricular breadth, the degree of variability in scope and 
level of learning across the system as a whole and within 'selected sub- 
systems, and others. 

A precedent for this kind of general assessment is found in the 
National Assessment of Educational Progress (NAEP), in which nationally 
representative samples of students at various age levels as well as young 
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adult* complete test** of general kinds of learning. That program requires 
an extensive special- administration of teits devised centrally by the 
NAEP. Similar results could be obtained wiuh examination questions devel- 
oped and administered by local faculty but shared with other faculty members. 
The costs would be limited to two processes— (1) coordination among the 
faculty members involved and (2) interpretation of the aggregated results. 

A major aspect of any assessment, process is the interpreation of the 
information it provides for the desired purposes. Grades, for example, 
are difficult to interpret across institutions.' Standardized tests provide 
information that is comparable across institutions in the assessment of 
students' current knowledge but that may not be appropriate for the eval- 
uation of institutional effects. The aggregation of results from selected 
groups of courses can permit program and institutional self-assessment 
that is pertinent to local curricula and objectives and that can be inter- 
preted in relation to several explicitly defined reference groups. 

The qiiality of higher education in the United States need not remain 
the mystery it has been despite decades of study. Its accomplishments cab 
be geucr^bed directly in terms of the widely diversified intellectual 
capabilities its graduates take with them. Components at various levels 
within the system can f£so be examined for direct indications of educa- 
tional quality. Continued reliance on intuition, wishful thinking, and 
unsupported rhetoric is indefensible. 
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